Skip to content

v0.4: historical trends — leaderboard + sparklines + headroom overflow projection#3

Merged
widgetii merged 1 commit into
mainfrom
feat/historical-trends
Jun 5, 2026
Merged

v0.4: historical trends — leaderboard + sparklines + headroom overflow projection#3
widgetii merged 1 commit into
mainfrom
feat/historical-trends

Conversation

@widgetii
Copy link
Copy Markdown
Member

@widgetii widgetii commented Jun 5, 2026

Summary

v0.2 shipped per-build sizes shards; v0.3 added the Kconfig configurator;
v0.4 finally reads across the retention window. Drift catches "package X
grew between two specific nightlies" — useful but you need to already
suspect a regression to pick the right two builds. TrendsView watches
the slope
— if a package has been growing 4 KB/week for 14 days, it
surfaces at the top of a leaderboard before the rootfs cap overflows.

Direct answer to PR OpenIPC/firmware#2163's
pain point — "~12 KB of accumulated drift since 2026-05-17 has now tipped
it over." With historical trends, the bisect step that PR had to do becomes
visible from the dashboard.

Data shape

`scripts/prebuild.mts` walks every (build × platform) sizes shard once
per source after the per-tag download loop, accumulates per-platform
series, writes:

```
public/data//trends/trends..json
```

Schema (matches `TrendsFile` in `src/lib/timeseries.ts` verbatim):

```ts
{
schema: 1,
source, platform, generated_at,
packages: { : [{build_id, built_at, bytes}, ...] },
modules: { : [{build_id, built_at, bytes}, ...] },
headroom_rootfs: [{build_id, built_at, used_kb, cap_kb, headroom_kb}, ...],
headroom_kernel: [{...}, ...]
}
```

Series sorted ascending by `built_at`. Defensive against partial shards
(missing `headroom` is skipped per-build, not fatal). `FsHooks` gains
`read` so the test `memFs` can serve readback inside the same in-memory
state the gh-download mock writes into.

Per-platform file at full retention: ~80 KB raw / ~10 KB gzipped (plan
estimate). Today's ~2-nightly state: ~16 KB raw / ~1.5 KB gzipped per
platform.

UI

A new Trends tab between Drift and Configure:

  • Header summary — builds count + date range covered.
  • HeadroomChart × 2 (rootfs + kernel) — draws the used-bytes curve
    against the cap line. When `projectOverflow(...)` returns a non-null
    projection within 60 days, an amber badge surfaces
    "projected overflow in 7d (2026-06-12)". Linear regression: minimise
    `(kb − (slope·day + intercept))²`, solve for `kb=0`. Sub-zero slope
    only — flat/growing returns null.
  • Controls strip — window selector (7/14/30/90d), packages|modules
    toggle, "min weekly Δ KB" filter.
  • Growers leaderboard — top-20 by absolute byte delta in the
    selected window. Inline 180×28 SVG sparkline per row (amber=grow,
    green=shrink), click row → expand 760×140 sparkline with axis labels.

Sparklines are hand-rolled SVG — no chart lib. `d3-hierarchy` is
already in the bundle (treemap) but doesn't transitively ship
`d3-scale` / `d3-shape`; rather than add ~20 KB of charting deps for
polylines, components pad + scale manually. Drop-in swap to a charting
lib stays easy if interactive zoom becomes a thing.

Tests

`tests/trends.test.ts` (12 cases):

  • `growthInWindow` — window filtering, sort tolerance, first/last
    semantics, < 2 points → null
  • `bytesPerDay` — window-endpoint slope, insufficient data → null
  • `topGrowers` — absolute-delta ordering, limit, skip insufficient,
    per-day rate
  • `projectOverflow` — flat/growing returns null; shrinking-at-known
    rate gives expected days-to-zero

`tests/bundle.test.ts` extension: no JS chunk embeds
`releases/download/...trends...` URLs (same regression guard as v0.3's
kconfig assertion).

`tests/prebuild.test.ts` memFs updated with `read`; defensive guards
in `emitTrends` let minimal-shard fixtures pass.

Numbers

v0.3 (shipped) v0.4 (this PR)
Active tests 46 63 (+12 trends, +adjustments)
JS bundle 172 KB / 55 KB gz 181 KB / 58 KB gz (+9 KB)
Initial launch ~60 KB gz unchanged
Per-platform trends ~16 KB raw / ~1.5 KB gz (today, 2 builds)
dist/data/firmware/trends 1.8 MB raw total

Under the 250 KB perf budget.

Test plan

  • `npm test` — 63 active pass, 3 live skipped
  • `npm run build` — clean, bundle invariants pass, 96 trends files
    emitted
  • Sample shape check on `hi3518ev300-lite`: schema 1, 31 packages
    × 2 builds of data, headroom_rootfs has 2 entries
  • Dispatch workflow on this branch (test+build green, deploy
    correctly rejected by main-only branch policy) before merge
  • Post-merge cron deploy populates Pages; live tests opt-in confirm
    production state
  • Trends tab renders the "only 2 points" placeholder today, gets
    richer as retention fills toward the ~2026-09-01 90-day mark

🤖 Generated with Claude Code

…droom overflow projection

v0.2 shipped per-build sizes shards; v0.3 added the Kconfig configurator;
v0.4 finally reads across the retention window. The Drift tab catches
"package X grew between two specific nightlies" — useful but you need
to already suspect a regression to pick the right two builds. TrendsView
watches the slope: if a package has been growing 4 KB/week for the last
14 days, it surfaces at the top of a leaderboard before the rootfs cap
overflows.

This is the direct answer to PR OpenIPC/firmware#2163's pain point —
"~12 KB of accumulated drift since 2026-05-17 has now tipped it over."
With historical trends, the bisect step that PR had to do becomes
visible from the dashboard.

Data shape

  scripts/prebuild.mts now walks every (build × platform) sizes shard
  once per source after the per-tag download loop, accumulates
  per-platform series into one PlatformAccumulator, and writes a
  per-platform aggregated file at:

      public/data/<source>/trends/trends.<platform>.json

  Schema (matches src/lib/timeseries.ts TrendsFile verbatim):

      {
        schema: 1,
        source, platform, generated_at,
        packages: { <name>: [{build_id, built_at, bytes}, ...] },
        modules:  { <name>: [{build_id, built_at, bytes}, ...] },
        headroom_rootfs: [{build_id, built_at, used_kb, cap_kb, headroom_kb}, ...],
        headroom_kernel: [{...}, ...]
      }

  Every series ships sorted by built_at ascending. Defensive against
  partial shards: missing `headroom` blocks are skipped per build, not
  fatal. New FsHooks.read so the test memFs can serve readback inside
  the same in-memory state the gh-download mock writes into.

Per-platform file: ~16 KB raw / ~1.5 KB gzipped on a board with 31
packages × 2 builds of data. Storage scales linearly with retention;
~80 KB raw at full 90-build retention. Total dist/data/firmware/trends/
across all 96 platforms today: 1.8 MB raw. Well under the plan's
~4 MB-gzipped envelope.

UI

  TrendsView (new "Trends" tab between "Drift vs another build" and
  "Configure (what-if)"):

    * Header summary: builds count + date range covered
    * Two HeadroomChart sections — rootfs and kernel — that draw the
      used-bytes curve against the cap line. When projectOverflow(...)
      returns a non-null projection within 60 days, an amber badge
      surfaces "projected overflow in 7d (2026-06-12)". Linear
      regression: minimise (kb - (slope*day + intercept))^2, solve for
      kb=0. Sub-zero slope only — flat/growing returns null (no
      overflow projected).
    * Controls strip: window selector (7/14/30/90d), packages|modules
      toggle, "min weekly delta KB" filter to surface only the
      noteworthy growers.
    * Growers leaderboard sorted by absolute byte delta in window,
      newest 20. Each row gets an inline Sparkline (180×28 SVG
      polyline, colour by direction: amber=grow, green=shrink), plus
      a click-to-expand large sparkline (760×140) with axis labels.

  Sparklines are hand-rolled SVG — no chart lib. d3-hierarchy is
  already in the bundle (treemap) but doesn't transitively ship
  d3-scale/d3-shape; rather than add ~20 KB of charting deps for
  polylines and axes, the component pads + scales manually. Drop-in
  swap to a charting lib is easy if interactive zoom becomes a thing
  later.

Tests

  tests/trends.test.ts (12 cases):
    * growthInWindow: window filtering, sort-on-input-order tolerance,
      first/last point semantics, < 2 points → null
    * bytesPerDay: window-endpoint slope, insufficient data → null
    * topGrowers: absolute-delta ordering, limit, skip insufficient,
      per-day rate
    * projectOverflow: flat/growing returns null, shrinking-at-known
      rate gives the expected days-to-zero

  tests/bundle.test.ts gains the same-origin invariant for
  trends/...json URLs — the regression guard for the original v0.1
  CORS bug class now covers trends URLs too.

  tests/prebuild.test.ts memFs updated with `read` so the runPrebuild
  cases still pass; defensive `headroom?.rootfs` guards in emitTrends
  let the minimal-shard fixtures run through trends emission without
  exploding.

Numbers

  - 63 active tests (was 46 in v0.3, +12 trends + adjustments)
  - JS bundle: 181 KB raw / 58 KB gzipped (+9 KB vs v0.3; under 250 KB
    perf budget)
  - Initial launch byte budget unchanged: ~60 KB gzipped (trends are
    lazy-loaded on Trends-tab click, ~1.5 KB gzipped per platform)
  - 96 trends files emitted today; 1.8 MB raw total

Roadmap from README

  v0.4 (this) shipped → v0.5 (build-request flow, optional, depends on
  v0.3 fragment) remains. Maintenance backlog (cache eviction, 404 UX,
  drift URL state, picker search, data-window header indicator)
  unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@widgetii widgetii merged commit 523a333 into main Jun 5, 2026
2 of 3 checks passed
@widgetii widgetii deleted the feat/historical-trends branch June 5, 2026 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant